Image compositing involves mixing several video (or film) sources, which may be photographic or graphical, to form a single composite. Traditional movie “matting” is the process of compositing two different film elements by printing them, one at a time, onto a duplicate strip of film. After one component is printed on the duplicate, the film is re-wound and the other component is added. Since the film cannot be exposed twice without creating a double exposure, the blank second area must be masked while the first is printed; then the freshly exposed first area must be masked while the second area is printed. Each masking is performed by a “traveling matte”—a specially altered duplicate shot which lies on top of the copy film stock.
The most common technique used for live action matting is known as chroma key. Foreground objects are placed in front of a screen of a selected reference color, usually green or blue, and imaged. Chroma keying can be done with backgrounds of any color that are uniform and distinct, but green and blue backgrounds (i.e., “green screen” and “blue screen”) are more commonly used because they differ most distinctly in hue from most human skin colors. The matte in this case is generated by color-based segmentation, which isolates the foreground objects from their background using the color as a data. This allows a new background to be substituted for the colored background. When chroma keying is used in video production and post-production, two images or video streams are overlaid based on color hues (chroma range).
Chroma keying has been widely used in various fields to remove a background from the subject of a photo or video—particularly in the news/weathercasting, motion picture and videogame industries. Commercially available computer software, such as Final Cut Pro, Pinnacle Studio, Adobe After Effects, and other programs now allow non-professionals to create videos with composited overlays using a chroma-key function and inexpensive green or blue screen kits.
The chroma-key technique has several disadvantages, however, one being the need for a special, uniformly colored background. Another problem is that if foreground objects contain colors similar to the color used for keying the background will show through. Further, using a single background color, the segmentation of the frame is limited to two layers only: a foreground layer, and (a colored) background layer that will be replaced completely by a new image; no further separation is possible between background and foreground objects.
Digital matting has replaced traditional, film-based approaches for several reasons. First, the required multiple strips of film may drift out of registration, resulting in halos and other undesirable edge artifacts. Digital matting is much more accurate, with precise alignment down to the pixel level in some cases. Digital images can be copied without quality loss, and multi-layer digital composites based upon 3-D models are readily generated.
With the proliferation of lightweight, handheld digital video cameras and smartphones with built-in cameras, users endeavor to take photographs of themselves in different environments. FIG. 1A is a drawing of a person 102 using a smart phone 104 with camera 106 to capture a self-shot. Holding the phone (or camera) at arm's length, the used can preview the image, including background features 108 on display screen 110. Once the desired composition is achieved, shutter button 112 is depressed or touched, and the image shown in FIG. 1B is captured. Among the problems with this approach is that all such “selfie” shots are similar in appearance, and may include a portion of the user's outstretched arm 114.